Image processing device, image processing method, and program

ABSTRACT

An image processing device includes a texture direction determining unit that determines a texture direction of an image, a defective pixel detecting unit that calculates a pixel value average for each of pixel groups including a plurality of pixels, and detects a defective pixel position on the basis of difference information of the pixel value average according to an arrangement direction of the pixel groups, and a correction unit that corrects, as a correction target, a pixel value at the defective pixel position detected on the basis of the difference information in the same pixel group arrangement direction as the texture direction determined by the texture direction determining unit.

BACKGROUND

The present disclosure relates to an image processing device, an imageprocessing method, and a program, and more particularly, to an imageprocessing device, an image processing method, and a program, whichcorrect a signal output from a defective pixel included in a solid-stateimaging device to suppress image quality deterioration caused by thedefective pixel.

Generally, in a solid-state imaging element such as a CCD (ChargeCoupled Device) and a CMOS (Complementary Metal Oxide Semiconductor)sensor, defective pixels may be included.

That is, in such a solid-state imaging element, a defective pixel may becaused by a partial crystal defect of a semiconductor and output anabnormal imaging signal, thereby causing image quality deterioration.For example, there is a black spot image defective pixel or a white spotdefective pixel. Various signal processing methods and circuitconfigurations have been proposed for correcting a signal output fromthe defective pixel.

In the related art disclosing a technique of correcting a signal outputfrom a defective pixel, there are, for example, Japanese UnexaminedPatent Application Publication No 2009-290653, Japanese Patent No.4343988, and Japanese Patent No. 4307318.

SUMMARY

In the related art described above, however, when detective pixelscontinuously occur or when a defect occurs at a circuit part (forexample, a transistor) shared by a plurality of pixels and defects occurin the plurality of pixels, it is difficult to acquire a texturedirection accurately and therefore it is difficult to sufficientlycorrect the defective pixels.

When the correction process disclosed in the related art is applied,only the center pixel is to be corrected in a neighboring area used inthe defect correction. For example, when a signal process is performedat a later stage using the neighboring area used for the defectcorrection and the defect is included in pixels other than the centerpixel, the defective pixel which has not been corrected may influencethe signal process of the later stage and it is difficult to obtain thecorrection effect. Accordingly, in the method of the related art, it isnecessary to perform the signal process of the later stage after thedefect correction is performed on all the pixels once, which causes aproblem of an increase of a circuit scale or a delay of a process speed.

It is desirable to provide an image processing device, an imageprocessing method, and a program that perform a determination process inwhich an influence of a defect is reduced when determining a directionof a texture so as to efficiently perform detection and correction of adefect even when there are a plurality of defective pixels in theneighboring area.

According to a first embodiment of the present disclosure, there isprovided an image processing device including a texture directiondetermining unit that determines a texture direction of an image, adefective pixel detecting unit that calculates a pixel value average foreach of pixel groups including a plurality of pixels and detects adefective pixel position on the basis of difference information of thepixel value average according to an arrangement direction of the pixelgroups, and a correction unit that corrects, as a correction target, apixel value at the defective pixel position detected on the basis of thedifference information in the same pixel group arrangement direction asthe texture direction determined by the texture direction determiningunit.

In the image processing device according to the embodiment of thepresent disclosure, the defective pixel detecting unit may calculate thepixel value average for each pixel group including the plurality ofpixels sharing a pixel value reading circuit and detect the defectivepixel position on the basis of the difference information of the pixelvalue average according to the arrangement direction of the pixel group.

In the image processing device according to the embodiment of thepresent disclosure, the defective pixel detecting unit may determinewhether or not a pixel is a defective pixel in accordance with adifference value from a reference value (safe_mW), the reference valuebeing an average value of a plurality of pixel groups in a flat areawhere a difference from the average value of a plurality of pixel groupunits is small in a plurality of adjacent arrangements of pixel groupsin the same arrangement direction.

In the image processing device according to the embodiment of thepresent disclosure, the texture direction determining unit may perform aprocess of determining one of the four directions of the horizontal,vertical, upper right, and lower right directions as the texturedirection, and the defective pixel detecting unit may detect thedefective pixel position on the basis of the difference information ofthe pixel value average according to the arrangement direction of thepixel groups in the four directions of horizontal, vertical, upperright, and lower right.

In the image processing device according to the embodiment of thepresent disclosure, the texture determining unit may calculate aplurality of differential values based on pixel values of pixelsarranged in a predetermined direction included in a neighboring areacentered on a pixel for attention, sort the plurality of differentialvalues, select only data with small values, calculate statistics, anddetermine the texture direction on the basis of comparison of thestatistics.

In the image processing device according to the embodiment of thepresent disclosure, the correction unit performs a process ofdetermining the pixel value at the defective pixel position withreference to a neighboring pixel in the texture direction as a referencepixel on the basis of the pixel value of the reference pixel.

In the image processing device according to the embodiment of thepresent disclosure, as a process target, the defective pixel detectingunit may calculate a pixel value average for each pixel group includinga plurality of pixels in the same pixel group arrangement direction asthe texture direction determined by the texture direction determiningunit, and detect the defective pixel position on the basis of thedifference information of the pixel value averages according to thearrangement direction of the pixel groups.

According to a second embodiment of the present disclosure, there isprovided an image processing method performed in an image processingdevice, the method including causing a texture direction determiningunit to determine a texture direction of an image, causing a defectivepixel detecting unit to calculate a pixel value average for each ofpixel groups including a plurality of pixels and to detect a defectivepixel position on the basis of difference information of the pixel valueaverage according to an arrangement direction of the pixel group, andcausing a correction unit to correct, as a correction target, thedefective pixel at the defective pixel position detected on the basis ofthe difference information in the same pixel group arrangement directionas the texture direction determined by the texture direction determiningunit.

According to a third embodiment of the present disclosure, there isprovided a program for causing an image processing device to execute animage process including causing a texture direction determining unit todetermine a texture direction of an image, causing a defective pixeldetecting unit to calculate a pixel value average for each of pixelgroups including a plurality of pixels and to detect a defective pixelposition on the basis of difference information of the pixel valueaverage according to an arrangement direction of the pixel group, andcausing a correction unit to correct, as a correction target, thedefective pixel at the defective pixel position detected on the basis ofthe difference information in the same pixel group arrangement directionas the texture direction determined by the texture direction determiningunit.

The program according to the embodiment of the present disclosure is,for example, a program which can be provided with a computer-readablestorage medium or communication medium usable in a general-purposesystem which can execute a variety of program code. Such acomputer-readable program is provided to perform a process according tothe program on a computer system.

Other characteristics and advantages of the present disclosure will beclarified by detailed description based on examples of the presentdisclosure and the accompanying drawings. In the specification, a systemis referred to as a logical group configuration of a plurality ofdevices, and the devices are not necessarily provided in the samecasing.

According to an embodiment of the present disclosure, a configuration ofperforming detection and correction of influence of a defective pixel ona captured image is realized. Specifically, a texture direction of animage is determined, a pixel value average is calculated for each pixelgroup including a plurality of pixels, and a defective pixel position isdetected on the basis of difference information of the pixel valueaverage according to an arrangement direction of the pixel group. Apixel value at the defective pixel position detected in the same pixelgroup arrangement direction as the texture direction is corrected as acorrection target. Defective pixel detection is performed at a positionin the texture direction, for example, on each pixel group sharing areading circuit to efficiently detect the defective pixel position.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a configuration of animage processing device according to an embodiment of the presentdisclosure.

FIG. 2 is a diagram illustrating an example of a configuration of animage processing unit of the image processing device according to theembodiment of the present disclosure.

FIG. 3 is a flowchart illustrating a process sequence performed by theimage processing device according to the embodiment of the presentdisclosure.

FIG. 4 is a diagram illustrating an example of a configuration of adefective pixel correcting process of the image processing deviceaccording to the embodiment of the present disclosure.

FIG. 5 is a diagram illustrating a texture direction determining processperformed in the defective pixel correction process of the imageprocessing device according to the embodiment of the present disclosure.

FIG. 6 is a diagram illustrating a horizontal direction differentialvalue calculating process performed in the texture direction determiningprocess performed by the image processing device according to theembodiment of the present disclosure.

FIG. 7 is a diagram illustrating a vertical direction differential valuecalculating process performed in the texture direction determiningprocess performed by the image processing device according to theembodiment of the present disclosure.

FIG. 8 is a diagram illustrating an upper right direction differentialvalue calculating process performed in the texture direction determiningprocess performed by the image processing device according to theembodiment of the present disclosure.

FIG. 9 is a diagram illustrating a lower right direction differentialvalue calculating process performed in the texture direction determiningprocess performed by the image processing device according to theembodiment of the present disclosure.

FIG. 10 is a diagram specifically illustrating a process performed by adefective pixel detecting unit in a defect correcting unit of the imageprocessing device according to the embodiment of the present disclosure.

FIG. 11 is a diagram illustrating an example of a process performed bythe defective pixel detecting unit in the defect correcting unit of theimage processing device according to the embodiment of the presentdisclosure.

FIG. 12 is a diagram illustrating a modified example of the defectivepixel detecting unit in the defect correcting unit of the imageprocessing device according to the embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, an image processing device, an image processing method, anda program according to an embodiment of the present disclosure will bedescribed in detail with reference to the drawings. An embodimentdescribed herein is implemented by an imaging device system. First, aconfiguration and an operation of an overall system will be described,and a process according an embodiment of the present disclosure will bedescribed in detail. The description is provided as follows.

1. Example of Configuration of Image Processing Device

2. Details of Image Process according to Embodiment of PresentDisclosure

3. Details of Configuration and Process of Defect Correcting Unit

4. Details of Process performed by Neighboring Area Extracting Unit inDefect Correcting Unit

5. Details of Process performed by Texture Direction Determining Unit inDefect Correcting Unit

6. Details of Process performed by Defective Pixel Detecting Unit inDefect Correcting Unit

7. Process of Defective Pixel Correcting Unit

1. Example of Configuration of Image Processing Device

FIG. 1 is an overall diagram of an imaging device (digital video camera)that is an example of an image processing device according to anembodiment of the present disclosure. The imaging device mainly includesan optical system, a signal processing system, a recording system, adisplay system, and a control system. Incident light passing through theoptical system including a lens and the like reaches an imaging element101 such as CMOS. The light first reaches light receiving elements of aCMOS imaging face, and is converted into an electrical signal byphotoelectric conversion in the light receiving element. Noise isremoved by a correlation double sampling circuit (CDS) 102, the signalis converted into digital data by a digitizing process of ananalogue-to-digital (A/D) converter 103, then the digital data istemporarily stored in an image memory of a DSP 104, and various signalprocesses are performed in the DSP 104.

In the imaging state, a timing generator (TG) 114 controls the signalprocessing system to keep image reading at a regular frame rate.

The A/D convertor 103 outputs a pixel stream at a regular rate to theDSP 104 where various image processes are performed, and then the imagedata is transmitted to an LCD driver 112, a CODEC 105, or both of them.The LCD driver 112 converts the image data transmitted from the DSP 104into an analog signal, and the analog signal is output to and displayedon an LCD 113. For example, the LCD 113 serves as a finder of a camera.The CODEC 105 performs encoding of the image data transmitted from theDSP 104, and the encoded image data is recorded in a memory 106.

The memory 106 may be a recording device using a semiconductor, amagnetic recording medium, an optical magnetic recording medium, anoptical recording medium, and the like.

For example, a CPU 115 performs an overall process control of an imagingprocess and an image process according to a program stored in a storageunit in advance.

An input unit 116 is an operation unit operated by a user.

The above description is description of an overall system of a digitalvideo camera of the embodiment.

In the imaging device shown in FIG. 1, for example, the DSP 104 mainlyperforms the processes according to an embodiment of the presentdisclosure. Hereinafter, an image process according to an embodiment ofthe present disclosure will be described in detail. In the followingembodiment, the DSP 104 performs the image process according to anembodiment of the present disclosure, but other hardware or softwarethan the DSP 104 may be used to perform the process according to anembodiment of the present disclosure. The other constituent element, forexample, the CMOS 101 may perform the image process.

2. Details of Image Process According to Embodiment of PresentDisclosure

As described above, for example, the image process according to anembodiment of the present disclosure can be performed by the DSP 104.

Accordingly, in the configuration of the embodiment describedhereinafter, an example will be described in which an operation unit inthe DSP 104 sequentially performs operations in the image process inaccordance with predetermined program code on a stream of the imagesignal input to the DSP 104.

In the embodiment described hereinafter, each process unit in theprogram will be described as a functional block, and a sequence ofperforming each process will be described using a flowchart. However, anembodiment of the present disclosure may be implemented by hardware, forexample, by mounting a hardware circuit realizing a process equivalentto the process executed by the functional block described hereinafter,in addition to being implemented by the form of the program described inthe embodiment.

FIG. 2 is a block diagram illustrating an example of a configuration ofan image processing unit performing the image process according to anembodiment of the present disclosure. As described above, the imageprocessing unit is configured by, for example, the DSP 104 shown inFIG. 1. In FIG. 2, mosaic image 117, Y image 124, and C image 125represented by two parallel horizontal lines indicate data or memoriesstoring data, and the other configurations including defect correctingunit 118 to YC conversion unit 123 represent processes performed in theimage processing unit or process units.

As shown in FIG. 2, the image processing unit includes a defectcorrecting unit 118, a white balance unit 119, a demosaic unit 120, amatrix unit 121, a gamma correcting unit 122, and a YC conversion unit123. The mosaic image 117 represents an input image to the imageprocessing unit, that is, an image signal input to the DSP 104 digitizedby the A/D convertor 103 shown in FIG. 1.

The mosaic image 117 is provided by storing an intensity signal (pixelvalue) of any of colors R, G, and B in a corresponding pixel of theimaging element 101 as shown in FIG. 1, and the color arrangement is,for example, a primary color system Bayer arrangement.

The Y image 124 and the C image 125 are images output from the imageprocessing unit. Those images correspond to a YCbCr image signal outputfrom the DSP 104 and input to the CODEC 105 as shown in FIG. 1.

The processes performed by the units of the image processing unit shownin FIG. 2 will be described.

The defect correcting unit 118 corrects a pixel value of a defectivepixel position into an accurate value in the mosaic image 117 input fromthe A/D converter 103 shown in FIG. 1.

A white balance unit 119 sets a proper coefficient in response to acolor of each pixel intensity such that a color balance of an achromaticcolor photography subject area is an achromatic color with respect tothe defect-corrected mosaic image.

The demosaic unit 120 performs an interpolation process so as to havethe same intensities of R, G, and B at the pixel positions of the mosaicimage subjected to the white balance adjustment. Outputs from thedemosaic unit 120 are three images in which pixel values of three colorsof R, G, and B are individually set at the pixel positions.

The matrix unit 121 applies a 3-row and 3-column linear matrix to whicha coefficient is set in advance to the pixels [R, G, and B] of theoutputs of the demosaic unit 120, and converts them into pixel values ofthree primary colors (intensity values R_m, G_m, and B_m). Thecoefficient of the linear matrix is an important design item to exhibitoptimal color representation, but the embodiment of the presentdisclosure relates to the defect correcting process. The matrix processis applied after the defect correcting process, and thus a specificvalue of a linear matrix coefficient may be designed irrespective of theembodiment of the present disclosure.

The outputs of the matrix unit 121 are three images corresponding tothree colors of color-corrected R_m, G_m, and B_m. After the matrixprocess, the gamma correcting unit 122 performs gamma correction on acolor-corrected 3-channel image.

The YC conversion unit 123 performs a YC matrix process and bandrestriction for a chroma component on the gamma-corrected 3-channelimage to generate the Y image 124 and the C image 125.

Next, a sequence of the process performed by the image processing unitshown in FIG. 2 will be described with reference to a flowchart shown inFIG. 3.

First, in Step S101, the image processing unit acquires a mosaic imagebased on an output signal of the image processing element 101. The imageis the mosaic image 117 shown in FIG. 2.

Then, in Step S102, the defect correcting unit 118 performs the defectcorrecting process on the mosaic image.

Then, in Step S103, the white balance unit 119 performs the whitebalance process on the defect-corrected mosaic image.

Then, in Step S104, the demosaic unit 120 performs the demosaic processof setting all the intensities (pixel values) of R, G, and B at thepixel positions of the mosaic image subjected to the white balanceprocess.

Then, in Step S105, the matrix unit 121 applies the linear matrix to thepixels of the 3-channel images and obtains an RGB 3-channel image.

Then, in Step S106, the gamma correcting unit 122 performs the gammacorrection on the pixels of the 3-channel image color-corrected by thematrix process.

Then, in Step S107, the YC conversion unit 123 performs YC conversion onthe gamma-corrected 3-channel image to generate the Y image 124 and theC image 125.

Last, in Step S108, the generated Y image 124 and C image 125 areoutput.

As described above, the operation of the image processing unit iscompleted.

3. Details of Configuration and Process of Defect Correcting Unit

Next, the image correcting process which is a main part of theembodiment of the present disclosure and is performed in the defectcorrecting unit 118 will be described in detail.

FIG. 4 shows a block diagram illustrating an internal configuration ofthe defect correcting unit 118.

As shown in FIG. 4, the defect correcting unit 118 mainly includes aneighboring area extracting unit 201, a texture direction determiningunit 202, a defective pixel detecting unit 203, and a defective pixelcorrecting unit 204.

The neighboring area extracting unit 201 cuts out a neighboring area 211in specific size including a position of a pixel for attention and itsneighboring area, from the mosaic image 117 input to the defectcorrecting unit 118, that is, the mosaic image 117 based on the outputsignal of the imaging element 101 shown in FIG. 1. In the embodiment,the neighboring area 211 is a rectangular area of 7×7 pixels centered onthe position of the pixel for attention.

The texture direction determining unit 202 determines a direction ofperforming the process of detecting the defective pixel in a pluralityof directions, at the position of the pixel for attention set to thecenter position of the rectangular area of 7×7 pixels. In theembodiment, the plurality of determination directions of the texturedirection determining unit 202 are four directions of:

Horizontal Direction (H direction),

Vertical Direction (V direction),

Upper Right Direction (A direction), and

Lower Right Direction (D direction).

The defective pixel detecting unit 203 performs the detection of thedefective pixel according to the direction of performing the process ofdetecting the defective pixel determined by the texture directiondetermining unit 202.

The defective pixel correcting unit 204 corrects the pixel value of thedefective pixel position detected by the defective pixel detecting unit203, using the pixels of the neighboring area 211.

The defect correcting unit 118 performs the correction of the defectivepixel by such a series of processes. Hereinafter, an example of specificprocesses of the process units constituting the defect correcting unit118 will be sequentially described.

4. Details of Process Performed by Neighboring Area Extracting Unit inDefect Correcting Unit

First, details of the process performed by the neighboring areaextracting unit 201 in the defect correcting unit 118 will be described.

The neighboring area extracting unit 201 performs an operation ofsecuring an access to pixel information in the 7×7 rectangular area inthe vicinity of the position of the pixel for attention. As a specificmethod thereof, various methods can be applied. For example, when theembodiment of the present disclosure is realized as software, the pixelvalues in the neighboring 7×7 rectangular area centered on the positionof the pixel for attention are secured in a memory in the form ofarrangement in which the pixel values are associated with, for example,coordinate positions.

When the embodiment of the present disclosure is realized as hardware, asignal processing system of a general imaging device is oftenimplemented such that a signal from a sensor flows sequentially as datain a first-order series of pixel intensities with a horizontal line. Inthis case, generally, an access to the pixels of the horizontal lineadjacent in the vertical direction is secured using a delay line capableof storing the pixel intensities (pixel values) of one horizontal line.

At least six delay lines are prepared to secure the access to the 7×7rectangular area.

5. Details of Process performed by Texture Direction Determining Unit inDefect Correcting Unit

Next, details of the process performed by the texture directiondetermining unit 202 in the defect correcting unit 118 will bedescribed.

FIG. 5 is a diagram illustrating a detailed configuration and operationof the texture direction determining unit 202.

The texture direction determining unit 202 has the followingdifferential value calculating units performing pixel value analysis ofthe neighboring area (in the example, 7×7 pixel area) of the pixel forattention:

(1) Horizontal Direction Differential Value Calculating Unit 311calculating Differential Value in Horizontal Direction,

(2) Vertical Direction Differential Value Calculating Unit 312calculating Differential Value in Vertical Direction,

(3) Upper Right Direction Differential Value Calculating Unit 313calculating Differential Value in Upper Right Direction, and

(4) Lower Right Direction Differential Value Calculating Unit 314calculating Differential Value in Lower Right Direction.

In the embodiment, the neighboring area is an area of 7×7 pixelscentered on the pixel for attention, and analysis is performed on theneighboring area.

The texture direction determining unit 202 further includes statisticcalculating units 321 a to 321 d calculating statistics from thedifferential values calculated regarding four directions, and astatistic comparing unit 331 determining the direction of the texture inthe neighboring area 211 by comparing the statistics of the directions.

The process of the horizontal direction differential value calculatingunit 311 calculating the differential value in the horizontal directionwill be described with reference to the drawings.

FIG. 6 is a diagram illustrating the process of the horizontal directiondifferential value calculating unit 311. FIG. 6 shows the neighboringarea of 7×7 pixels centered on the pixel for attention.

The horizontal direction is represented by the x coordinate, and thevertical direction is represented by the y coordinate. The center of theneighboring area of 7×7 pixels, that is, (x, y)=(4, 4) corresponds tothe position of the pixel for attention.

The pixel used in the texture direction determination is represented byW. At the pixel position (x, y), the horizontal direction differentialvalue gradH(x, y) is acquired by the following formula:

gradH(x,y)=abs(w(x−1,y)−w(x+1,y))

where abs( ) is a function of acquiring an absolute value, w(x−1, y) isa pixel value (intensity) of W at the coordinate position (x−1, y), andw(x+1, y) is a pixel value (intensity) of W at the coordinate position(x+1, y).

The horizontal direction differential value calculating unit 311acquires the differential value gradH at the pixel positions indicatedby symbol O in FIG. 6.

For example, at the position of symbol O of the position of (x, y)=(2,1), the differential value gradH is acquired on the basis of the pixelvalues (intensities) of both neighboring W in the horizontal direction.

That is, the differential value gradH(2, 1) is calculated using thepixel values of two W pixels of (x, y)=(1, 1) and (x, y)=(3, 1).

The differential values gradH are acquired at eighteen pixel positionsindicated by symbol O in FIG. 6.

Next, the process of the vertical direction differential valuecalculating unit 312 calculating the differential value in the verticaldirection will be described.

FIG. 7 is a diagram illustrating the process of the vertical directiondifferential value calculating unit 312. FIG. 7 shows the neighboringarea of 7×7 pixels centered on the pixel for attention.

The horizontal direction is represented by the x coordinate, and thevertical direction is represented by the y coordinate. The center of theneighboring area of 7×7 pixels, that is, (x, y)=(4, 4) corresponds tothe position of the pixel for attention.

The pixel used in the texture direction determination is represented byW. At the pixel position (x, y), the vertical direction differentialvalue gradV(x, y) is acquired by the following formula:

gradV(x,y)=abs(w(x,y−1)−w(x,y+1))

where abs( ) is a function of acquiring an absolute value, w(x, y−1) isa pixel value (intensity) of W at the coordinate position (x, y−1), andw(x, y+1) is a pixel value (intensity) of W at the coordinate position(x, y+1).

The vertical direction differential value calculating unit 312 acquiresthe differential value gradV at the pixel positions indicated by symbolO in FIG. 7.

For example, at the position of symbol O of the position of (x, y)=(1,2), the differential value gradV is acquired on the basis of the pixelvalues (intensities) of both neighboring W in the vertical direction.

That is, the differential value gradV(1, 2) is calculated using thepixel values of two W pixels of (x, y)=(1, 1) and (x, y)=(1, 3).

The differential values gradV are acquired at eighteen pixel positionsindicated by symbol O in FIG. 7.

Next, the process of the upper right direction differential valuecalculating unit 313 calculating the differential value in the upperright direction will be described.

FIG. 8 is a diagram illustrating the process of the upper rightdirection differential value calculating unit 313. FIG. 8 shows theneighboring area of/pixels centered on the pixel for attention.

The horizontal direction is represented by the x coordinate, and thevertical direction is represented by the y coordinate. The center of theneighboring area of 7×7 pixels, that is, (x, y)=(4, 4) corresponds tothe position of the pixel for attention.

The pixel used in the texture direction determination is represented byW. At the pixel position (x, y), the upper right direction differentialvalue gradA(x, y) is acquired by the following formula:

gradA(x,y)=abs(w(x,y)−w(x+1,y−1))

where abs( ) is a function of acquiring an absolute value, w(x, y) is apixel value (intensity) of W at the coordinate position (x, y), andw(x+1, y−1) is a pixel value (intensity) of W at the coordinate position(x+1, y−1).

The upper right direction differential value calculating unit 313acquires the differential value gradA at the W pixel positions within adotted line in FIG. 8.

For example, at the position of symbol W of the position of (x, y)=(1,3), the differential value gradA is acquired on the basis of the pixelvalues (intensities) of the pixel itself and the upper right adjacent W.

That is, the differential value gradA(1, 3) is calculated using thepixel values of two W pixels of (x, y)=(1, 3) and (x, y)=(2, 2).

The differential values gradA are acquired at eighteen W pixel positionswithin a dotted line shown in FIG. 8.

Next, the process of the lower right direction differential valuecalculating unit 314 calculating the differential value in the lowerright direction will be described.

FIG. 9 is a diagram illustrating the process of the lower rightdirection differential value calculating unit 314. FIG. 9 shows theneighboring area of 7×7 pixels centered on the pixel for attention.

The horizontal direction is represented by the x coordinate, and thevertical direction is represented by the y coordinate. The center of theneighboring area of 7×7 pixels, that is, (x, y)=(4, 4) corresponds tothe position of the pixel for attention.

The pixel used in the texture direction determination is represented byW. At the pixel position (x, y), the lower right direction differentialvalue gradD(x, y) is acquired by the following formula:

gradD(x,y)=abs(w(x,y)−w(x+1,y+1))

where abs( ) is a function of acquiring an absolute value, w(x, y) is apixel value (intensity) of W at the coordinate position (x, y), andw(x+1, y+1) is a pixel value (intensity) of W at the coordinate position(x+1, y+1).

The lower right direction differential value calculating unit 314acquires the differential value gradD at the W pixel position within adotted line in FIG. 9.

For example, at the position of symbol W of the position of (x, y)=(1,1), the differential value gradD is acquired on the basis of the pixelvalues (intensities) of the pixel itself and the lower right adjacent W.

That is, the differential value gradD(1, 1) is calculated using thepixel values of two W pixels of (x, y)=(1, 1) and (x, y)=(2, 2).

The differential values gradD are acquired at eighteen W pixel positionswithin a dotted line shown in FIG. 9.

Next, the processes of the statistic calculating units 321 a to 321 dwill be described.

The statistic calculating units 321 a to 321 d perform sorting, on thebasis of the magnitudes of the differential values, on the differentialvalues gradH, gradV, gradA, and gradD respectively calculated by thedifferential value calculating units of:

(1) Horizontal Direction Differential Value Calculating Unit 311,

(2) Vertical Direction Differential Value Calculating Unit 312,

(3) Upper Right Direction Differential Value Calculating Unit 313, and

(4) Lower Right Direction Differential Value Calculating Unit 314.

Average values mHgrad, mVgrad, mAgrad, and mDgrad are calculated usingthe differential values to the n-th differential value in ascendingorder.

As described with reference to FIG. 6 to FIG. 9, the followingdifferential values are calculated by the pixel analysis of the 7×7neighboring area centered on one pixel for attention:

(1) Eighteen Horizontal Direction Differential Values gradH ofHorizontal Direction Differential Calculating Unit 311,

(2) Eighteen Vertical Direction Differential Values gradV of VerticalDirection Differential Calculating Unit 312,

(3) Eighteen Upper Right Direction Differential Values gradA of UpperRight Direction Differential Calculating Unit 313, and

(4) Eighteen Lower Right Direction Differential Values gradD of Rightdown Direction Differential Calculating Unit 314.

The statistic calculating units 321 a to 321 d perform sorting on thebasis of the magnitudes of the differential values as the statisticsbased on the plurality of differential values, and calculate the averagedifferential values mHgrad, mVgrad, mAgrad, and mDgrad using thedifferential values to the n-th value in ascending order.

Herein, n is a value equal to or less than the length N of the sorteddifferential values. In the example, N is 18.

The value of n is determined depending on the magnitude of a continuousdefect assumed to be included in the neighboring area 211. For example,when the continuous defective pixels included in the neighboring areapixels are 2×2 pixels, the differential values which are not accuratelyacquired by the influence of the defect are maximal 4 pixels, and thusn=N−4.

Next, the process of the statistic comparing unit 331 will be described.

The statistic comparing unit 331 compares the statistics mHgrad, mVgrad,mAgrad, and mDgrad calculated by the statistic calculating units 321 ato 321 d in the horizontal, vertical, upper right, and lower rightdirections as described above, and determines a direction with thesmallest statistic as the texture direction.

For example, when the mHgrad is the smallest value, the texturedirection of the neighboring area 211 is determined as the horizontaldirection.

The texture direction determining unit 202 in the defect correcting unit118 determines the texture direction as described above, and outputstexture direction information (dir) as determination information to thedefective pixel detecting unit 203.

The texture direction information (dir) corresponds to a direction withthe smallest brightness change or pixel value change.

6. Details of Process performed by Defective Pixel Detecting Unit inDefect Correcting Unit

Next, details of the process performed by the defective pixel detectingunit 203 in the defect correcting unit 118 will be described withreference to FIG. 10.

FIG. 10 is a diagram illustrating a process performed by the defectivepixel detecting unit 203 and a configuration thereof.

The defective pixel detecting unit 203 performs the process on thepixels constituting the neighboring area 211 (in the example, the areaof 7×7 pixels centered on the pixel for attention) for each sharingpixel group formed of a plurality of pixels sharing a pixel outputreading circuit (sharing FD). The sharing pixel group is a group ofpixels sharing the reading circuit.

The sharing pixel statistic calculating unit 411 calculates a statisticfor each sharing pixel group formed of a plurality of pixels sharing thepixel output reading circuit (sharing FD).

In addition, the defective pixel detecting unit 203 includes thefollowing units that use the statistics calculated by the sharing pixelstatistic calculating unit 411 and detect pixels estimated to bedefective in the respective directions in the neighboring area 211. Thatis, the following defect detecting units corresponding to fourdirections are provided:

(1) Horizontal Direction Defect Detecting Unit 421 performing DefectDetection in Horizontal Direction,

(2) Vertical Direction Defect Detecting Unit 422 performing DefectDetection in Vertical Direction,

(3) Upper Right Direction Defect Detecting Unit 423 performing DefectDetection in Upper Right Direction, and

(4) Lower Right Direction Defect Detecting Unit 424 performing DefectDetection in Lower Right Direction.

In addition, the defective pixel detecting unit 203 includes a defectivepixel position selecting unit 431 that inputs the texture directioninformation (dir) determined by the texture direction determining unit202 described above and selects a correction target defective pixelposition to be a correction target from the pixel estimated to be thedefective pixels by the defective detecting units 421 to 424 at thepreceding stage.

For example, as a sharing pixel configuration of the solid-state imagingdevice, there is a pixel configuration in units of 8 pixels.

FIG. 11 shows an example of a configuration in which 8 pixels connectedby a solid line indicate the sharing pixels.

That is, the example shown in FIG. 11 describes a configuration of asharing pixel group including 8 pixels and using one pixel outputreading circuit (sharing FD).

In the case of a sharing pixel pattern shown in FIG. 11, the pixel groupof the sharing pixels (8 pixels) including the pixel for attention isFD01.

Pixel groups of the sharing pixels in the neighboring left, lower left,down, lower right, and right directions of the pixel group of thesharing pixels FD01 including the pixel for attention are sequentiallyindicated by FD00, FD10, FD11, FD12, and FD22.

That is, in the 7×7 pixel area centered on the pixel for attention, thepixel groups of the sharing pixels (8 pixels) such as FD00, FD01, andFD02 from the upper left side and FD10, FD11, and FD12 from the lowerleft side are set.

When a pixel output reading circuit is shared by a sharing pixel group,there is a case where one pixel of the group is defective, or a casewhere an output value is set to a value away from a normal value due toa defect of the reading circuit. For example, all the constituent pixelsof the pixel group FD00 shown in FIG. 11 may be defective pixels.

Hereinafter, an example of a process when one of the pixel groups FD01and FD11 is defective will be described.

First, the sharing pixel statistic calculating unit 411 calculates apixel value average value of W pixels for each sharing pixel group inthe neighboring area 211. In the example, the neighboring area 211 isthe 7×7 pixel area centered on the pixel for attention.

For example, mW01 is an average value of the W pixels included in thesharing pixel group (FD01) including the center pixel surrounded by adotted line in the pixel area formed of 7×7 pixels shown in FIG. 11. Thesharing pixel group is a group of the pixels sharing the reading circuitas described above.

Similarly, mW00 is the average value of W pixels included in the pixelsharing group (FD00);

mW01 is the average value of W pixels included in the pixel sharinggroup (FD01);

mW02 is the average value of W pixels included in the pixel sharinggroup (FD02);

mW10 is the average value of W pixels included in the pixel sharinggroup (FD10);

mW11 is the average value of W pixels included in the pixel sharinggroup (FD11); and

mW12 is the average value of W pixels included in the pixel sharinggroup (FD12).

An average value group formed of six average values (mW00 to mW12) ofthe respective sharing pixel groups is an average value group 1.

That is, the average value group 1 is formed of the average values oftotal 2×3 pixel sharing groups: the pixel sharing group (FD01) includingthe pixel for attention (center pixel surrounded with the dotted line inthe 7×7 pixel area shown in FIG. 11) and the adjacent sharing pixelgroups of the left (FD00), lower left (FD10), down (FD11), lower right(FD12), and right (FD02) directions thereof.

The average value group 1 is calculated corresponding to the six sharingpixel groups FD00 to FD12 shown in FIG. 11. Alternatively, aconfiguration using an average value group 2 may be formed by using theaverage values of total 2×3 pixel sharing groups: the sharing pixelgroup (FD01) including the pixel for attention, and the adjacent sharingpixel groups of the left (FD00), upper left (not shown, above FD00), up(not shown, above FD01), upper right (not shown, above FD02), and right(FD02).

Whether to use the average value group 1 or the average value group 2 isdetermined depending on a phase of the pixel for attention and a patternof the sharing pixel. That is, the average value of the W pixels sharingthe FD00 is represented by mW00, and the average value of the W pixelssharing another FD is represented in the same manner.

The process of the horizontal direction defect detecting unit 421 willbe described. First, the horizontal direction defect detecting unit 421calculates horizontal direction difference values (gradients: gH00 togH11) of the following sharing pixel groups on the basis of 2×3 averagevalues (mW00 to mW12) calculated by the sharing pixel statisticcalculating unit 411 of the preceding stage. That is,gH00=abs(mW00−mW01), gH01=abs(mW01−mW02), gH10=abs(mW10−mW11), andgH11=abs(mW11−mW12) are calculated.

The difference values (gradients) correspond to first-order differentialvalues.

The horizontal direction defect detecting unit 421 calculates thefollowing difference average values on the basis of the horizontaldirection difference values (gradients: gH00 to gH11) of the respectivesharing pixel groups:

Average Value of Upper Horizontal Direction Difference Values(gradients: gH00 and gH01)

gH0=(gH00+gH01)/2

Average Value of Lower Horizontal Direction Difference Values(gradients: gH10 and gH11)

gH1=(gH10+gH11)/2

The horizontal direction defect detecting unit 421 compares thefollowing:

Difference Average Value (gH0=(gH00+gH01)/2) that is Average Value ofUpper Horizontal Direction Difference Values (gradients), and

Difference Average Value (gH1=(gH10+gH11)/2) that is Average Value ofLower Horizontal Direction Difference Values (gradients).

The horizontal direction defect detecting unit 421 calculates thehorizontal direction defect detecting average value (safe_mW) as followsin accordance with the comparison result.

(a) if gH0<gH1,

safe_(—) mW=(mW00+mW01+mW02)/3, and

(b) if gH0>gH1,

safe_(—) mW=(mW10+mW11+mW12)/3.

According to any of the above (a) and (b), the horizontal directiondefect detecting average value safe_mW is acquired.

The process is a process of selecting a flat area with a smaller changein pixel values from the upper and lower pixel areas, and calculatingthe average value as the horizontal direction defect detecting averagevalue (safe_mW).

Then, the horizontal direction defect detecting unit 421 determines thatthe sharing pixel group FD01 has a defect, when the difference averagevalue (gH0) of the upper horizontal direction difference values(gradients) of the upper sharing pixel groups including the sharingpixel group (FD01) of the defect determination target is equal to orlarger than a predetermined threshold value and the following conditionformula is satisfied:

abs(mW01−safe_(—) mW)>abs(mW00−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD01) of the defect determination target has a defect, whenthe difference average value (gH0) of the upper horizontal directiondifference values (gradients) of the upper sharing pixel groupsincluding the sharing pixel group (FD01) of the defect determinationtarget is equal to or larger than a predetermined threshold value andthe difference between the pixel value average (mW01) of the sharingpixel group (FD01) of the defect determination target and the horizontaldirection defect detecting average value (safe_mW) is larger than thedifference between the pixel value average (mW00) of the left sharingpixel group (FD00) adjacent to the sharing pixel group (FD01) in thehorizontal direction and the horizontal direction defect detectingaverage value (safe_mW).

When the difference average value (gH1) of the lower horizontaldirection difference values (gradients) of the lower sharing pixelgroups including the sharing pixel group (FD11) of the defectdetermination target is equal to or larger than a predeterminedthreshold value and the following condition formula is satisfied, thesharing pixel group (FD11) is determined to have a defect.

abs(mW11−safe_(—) mW)>abs(mW10−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD11) of the defect determination target has a defect, whenthe difference average value (gH1) of the lower horizontal directiondifference values (gradients) is equal to or larger than a predeterminedthreshold value and the difference between the pixel value average(mW11) of the sharing pixel group (FD11) of the defect determinationtarget and the horizontal direction defect detecting average value(safe_mW) is larger than the difference between the pixel value average(mW10) of the left sharing pixel group (FD10) adjacent to the sharingpixel group (FD11) and the horizontal direction defect detecting averagevalue (safe_mW).

Next, the process of the vertical direction defect detecting unit 422will be described.

The vertical direction defect detecting unit 422 calculates verticaldirection difference values (gradients: gV0 and gV1) of the followingsharing pixel groups on the basis of 2×3 average values (mW00 to mW12)calculated by the sharing pixel statistic calculating unit 411 of thepreceding stage. That is, the vertical direction defect detecting unit422 calculates the vertical direction difference value (gradient) of thesharing pixel groups (FD00 and FD10) of the left column: gV0,gV0=abs(mW00−mW10), and the vertical direction difference value(gradient) of the sharing pixel groups (FD01 and FD11) of the rightcolumn: gV1, gV1=abs(mW01−mW11).

Furthermore, the vertical direction defect detecting unit 422 comparestwo vertical direction difference values (gradients: gV0 and gV1) of therespective sharing pixel groups, and calculates the following verticaldirection defect detecting average value (safe_mW) on the basis of thecomparison result.

(a) if gV0<gV1,

safe_(—) mW=(mW00+mW10)/2, and

(b) if gV0>gV1,

safe_(—) mW=(mW01+mW11)/2.

According to any of the above (a) and (b), the vertical direction defectdetecting average value safe_mW is acquired.

The process is a process of selecting a flat area with a smaller changein pixel values from pixel areas of the columns of the sharing pixelgroups in two vertical columns adjacent to each other, that is, the leftcolumn including the sharing pixel groups (FD00 and FD10) and the rightcolumn including the sharing pixel groups FD01 and FD11, and calculatingthe average value as the vertical direction detect detecting averagevalue (safe_mW).

Then, the vertical direction defect detecting unit 422 determines thatthe sharing pixel group (FD01) has a defect, when the vertical directiondifference value (gradient) gV1 of the column including the sharingpixel group (FD01) of the defect determination target is equal to orlarger than a predetermined threshold value and the following conditionformula is satisfied:

abs(mW01−safe_(—) mW)>abs(mW11−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD01) of the defect determination target has a defect, whenthe vertical direction difference value (gradient) (gV1) of the verticaldirection column including the sharing pixel group (FD01) of the defectdetermination target is equal to or larger than a predeterminedthreshold value and the difference between the pixel value average(mW01) of the sharing pixel group (FD01) of the defect determinationtarget and the vertical direction defect detecting average value(safe_mW) is larger than the difference between the pixel value average(mW11) of the lower sharing pixel group (FD11) adjacent to the sharingpixel group (FD01) in the vertical direction and the vertical directiondefect detecting average value (safe_mW).

Then, the vertical direction defect detecting unit 422 determines thatthe sharing pixel group (FD11) has a defect, when the vertical directiondifference value (gradient) gV1 of the column including the sharingpixel group (FD11) of the defect determination target is equal to largerthan a predetermined threshold value and the following condition formulais satisfied:

abs(mW11−safe_(—) mW)>abs(mW01−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD11) of the defect determination target has a defect, whenthe vertical direction difference value (gradient) (gV1) of the verticaldirection column including the sharing pixel group (FD11) of the defectdetermination target is equal to or larger than a predeterminedthreshold value and the difference between the pixel value average(mW11) of the sharing pixel group (FD11) of the defect determinationtarget and the vertical direction defect detecting average value(safe_mW) is larger than the difference between the pixel value average(mW01) of the upper sharing pixel group (FD01) adjacent to the sharingpixel group (FD11) in the vertical direction and the vertical directiondefect detecting average value (safe_mW).

Next, the process of the upper right direction defect detecting unit 423will be described.

The upper right direction defect detecting unit 423 calculates upperright direction difference values (gradients: gA0 and gA1) of thefollowing sharing pixel groups on the basis of 2×3 average values (mW00to mW12) calculated by the sharing pixel statistic calculating unit 411of the preceding stage. That is, the upper right direction defectdetecting unit 423 calculates the upper right direction difference value(gradient) of the sharing pixel groups (FD10 and FD01) adjacent in theupper right direction: gA0, gA0=abs(mW10−mW01), and the upper rightdirection difference value (gradient) of the sharing pixel groups (FD11and FD02) adjacent in the upper right direction: gA1,gA1=abs(mW11−mW02).

The upper right direction defect detecting unit 423 compares two upperright direction difference values (gradients: gA0 and gA1) of therespective sharing pixel groups, and calculates the following upperright direction defect detecting average value (safe_mW) on the basis ofthe comparison result.

(a) if gA0<gA1,

safe_(—) mW=(mW10+mW01)/2, and

(b) if gA0>gA1,

safe_(—) mW=(mW11+mW02)/2.

According to any of the above (a) and (b), the upper right directiondefect detecting average value safe_mW is acquired.

The process is a process of selecting a flat area with a smaller changein pixel values from the adjacent data of sharing pixel groups in twoupper right direction lines, that is, the sharing pixel groups (FD10 andFD01) and the sharing pixel groups (FD11 and FD02), and calculating theaverage value as the upper right direction defect detecting averagevalue (safe_mW).

Then, the upper right direction defect detecting unit 423 determinesthat the sharing pixel group FD01 has a defect, when the upper rightdirection difference value (gradient) gA0 of the sharing pixel groupsincluding the sharing pixel group (FD01) of the defect determinationtarget is equal to or larger than a predetermined threshold value andthe following condition formula is satisfied:

abs(mW01−safe_(—) mW)>abs(mW10−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD01) of the defect determination target has a defect, whenthe upper right direction difference value (gradient) (gA1) in the upperright direction including the sharing pixel group (FD01) of the defectdetermination target is equal to or larger than a predeterminedthreshold value and the difference between the pixel value average(mW01) of the sharing pixel group (FD01) of the defect determinationtarget and the upper right direction defect detecting average value(safe_mW) is larger than the difference between the pixel value average(mW10) of the lower left sharing pixel group (FD10) adjacent to thesharing pixel group (FD01) in the upper right direction and the upperright direction defect detecting average value (safe_mW).

Then, the upper right direction defect detecting unit 423 determinesthat the sharing pixel group FD11 has a defect, when the upper rightdirection difference value (gradient) gA1 of the sharing pixel groups inthe upper right direction including the sharing pixel group (FD11) ofthe defect determination target is equal to or larger than apredetermined threshold value and the following condition formula issatisfied:

abs(mW11−safe_(—) mW)>abs(mW02−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD11) of the defect determination target has a defect, whenthe upper right direction difference value (gradient) (gA1) in the upperright direction including the sharing pixel group (FD11) of the defectdetermination target is equal to or larger than a predeterminedthreshold value and the difference between the pixel value average(mW11) of the sharing pixel group (FD11) of the defect determinationtarget and the upper right direction defect detecting average value(safe_mW) is larger than the difference between the pixel value average(mW02) of the upper right sharing pixel group (FD02) adjacent to thesharing pixel group (FD11) in the upper right direction and the upperright direction defect detecting average value (safe_mW).

Next, the process of the lower right direction defect detecting unit 424will be described.

The lower right direction defect detecting unit 424 calculates lowerright direction difference values (gradients: gD0 and gD1) of thefollowing sharing pixel groups on the basis of 2×3 average values (mW00to mW12) calculated by the sharing pixel statistic calculating unit 411of the preceding stage. That is, the lower right direction defectdetecting unit 424 calculates the lower right direction difference value(gradient) of the sharing pixel groups (FD00 and FD11) adjacent in thelower right direction: gD0, gD0=abs(mW00−mW11), and the lower rightdirection difference value (gradient) of the sharing pixel groups (FD01and FD12) adjacent in the lower right direction: gD1,gD1=abs(mW01−mW12).

The lower right direction defect detecting unit 424 compares two lowerright direction difference values (gradients: gD0 and gD1) of therespective sharing pixel groups, and calculates the following lowerright direction defect detecting average value (safe_mW) on the basis ofthe comparison result.

(a) if gD0<gD1,

safe_(—) mW=(mW00+mW11)/2, and

(b) if gD0>gD1,

safe_(—) mW=(mW01+mW12)/2.

According to any of the above (a) and (b), the lower right directiondefect detecting average value safe_mW is acquired.

The process is a process of selecting a flat area with a smaller changein pixel values from the adjacent data of sharing pixel groups in twolower right direction lines, that is, the pixel sharing groups (FD00 andFD11) and the pixel sharing groups (FD01 and FD12), and calculating theaverage value as the vertical direction defect detecting average value(safe_mW).

Then, the lower right direction defect detecting unit 424 determinesthat the pixel sharing group FD01 has a defect, when the lower rightdirection difference value (gradient) gD1 of the sharing pixel groups inthe lower right direction including the sharing pixel group (FD01) ofthe defect determination target is equal to or larger than apredetermined threshold value and the following condition formula issatisfied.

abs(mW01−safe_(—) mW)>abs(mW12−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD01) of the defect determination target has a defect, whenthe lower right direction difference value (gradient) (gD1) of thesharing pixel groups in the lower right direction including the sharingpixel group (FD01) of the defect determination target is equal to orlarger than a predetermined threshold value and the difference betweenthe pixel value average (mW01) of the sharing pixel group (FD01) of thedefect determination target and the lower right direction defectdetecting average value (safe_mW) is larger than the difference betweenthe pixel value average (mW12) of the lower right sharing pixel group(FD12) adjacent to the sharing pixel group (FD01) in the lower rightdirection and the lower right direction defect detecting average value(safe_mW).

Then, the lower right direction defect detecting unit 424 determinesthat the sharing pixel group FD11 has a defect, when the lower rightdirection difference value (gradient) gD0 of the sharing pixel groupsincluding the sharing pixel group (FD11) of the defect determinationtarget is equal to larger than a predetermined threshold value and thefollowing condition formula is satisfied:

abs(mW11−safe_(—) mW)>abs(mW00−safe_(—) mW)

The determination process is a process of determining that the sharingpixel group (FD11) of the defect determination target has a defect, whenthe lower right direction difference value (gradient) (gD0) of thesharing pixel groups in the lower right direction including the sharingpixel group (FD11) of the defect determination target is equal to orlarger than a predetermined threshold value and the difference betweenthe pixel value average (mW11) of the sharing pixel group (FD11) of thedefect determination target and the lower right direction defectdetecting average value (safe_mW) is larger than the difference betweenthe pixel value average (mW00) of the upper left sharing pixel group(FD00) adjacent to the sharing pixel group (FD11) in the lower rightdirection and the lower right direction defect detecting average value(safe_mW).

Next, the process of the defective pixel position selecting unit 431will be described.

The defective pixel position selecting unit 431 selects the detectionresult of the texture direction (dir) determined by the texturedirection determining unit 202 described above, from the pixel positionsdetected by the defect detecting units 421 to 424 of the respectivedirections described above.

That is, when the texture direction (dir) determined by the texturedirection determining unit 202 is the horizontal direction (H), thedetection result of the horizontal direction defect detecting unit 421is selected. That is, when the horizontal direction defect detectingunit 421 determines that the sharing pixel group FDxy (for example,FD01) has a defective pixel, the sharing pixel group FDxy is selected asthe sharing pixel group having the defective pixel of the correctiontarget.

When the texture direction (dir) determined by the texture directiondetermining unit 202 is the vertical direction (V), the detection resultof the vertical direction defect detecting unit 422 is selected. Thatis, when the vertical direction defect detecting unit 422 determinesthat the sharing pixel group FDxy (for example, FD01) has a defectivepixel, the sharing pixel group FDxy is selected as the sharing pixelgroup having the defective pixel of the correction target.

When the texture direction (dir) determined by the texture directiondetermining unit 202 is the upper right direction (A), the detectionresult of the upper right direction defect detecting unit 423 isselected. That is, when the upper right direction defect detecting unit423 determines that the sharing pixel group FDxy (for example, FD01) hasa defective pixel, the sharing pixel group FDxy is selected as thesharing pixel group including the defective pixel of the correctiontarget.

When the texture direction (dir) determined by the texture directiondetermining unit 202 is the lower right direction (D), the detectionresult of the lower right direction defect detecting unit 424 isselected. That is, when the lower right direction defect detecting unit424 determines that the sharing pixel group FDxy (for example, FD01) hasa defective pixel, the sharing pixel group FDxy is selected as thesharing pixel group including the defective pixel of the correctiontarget.

The selection process of the correction target defective pixel performedby the defective pixel position selecting unit 431 is a process ofdetermining that only a pixel at a position in a direction correspondingto the texture direction (dir) determined by the texture directiondetermining unit 202 has a high probability of being an actuallydefective pixel, from the pixels estimated as pixels having a defect bythe defect detecting units 421 to 424, and selecting the pixel as thecorrection target.

The selection process will be described.

In the defect detecting units 421 to 424, the sharing pixel group with alarge change in pixel values in the corresponding direction is set asthe sharing pixel group probably having a defective pixel, and the pixelgroup is determined as the group including the defective pixel.

However, the defective pixel determined by the defect detecting units421 to 424 may not be a defective pixel to be corrected and may output atrue value.

The defective pixel position selecting unit 431 selects a sharing pixelgroup including a defective pixel to be corrected from the sharing pixelgroups including pixels determined as the defective pixels by the defectdetecting units 421 to 423. For the selection process, the texturedirection information (dir) is applied.

The texture direction is originally a direction with a small change inpixel values.

In the defect detecting units 421 to 424, the sharing pixel group with alarge change in pixel values in the corresponding direction is estimatedas the group probably having a defective pixel, and these are determinedas the defective pixels.

The defective pixel position selecting unit 431 selects only a sharingpixel group corresponding to the texture direction from the outputs ofthe defect detecting units 421 to 424, as a sharing pixel group havingan actual defective pixel to be corrected. The other groups aredetermined to be highly probably outputting the actual pixel value andare excepted from the correction target.

In the configuration of the defective pixel detecting unit 203 describedwith reference to FIG. 10, the pixel group estimated to have a defectivepixel is determined in the detecting units of the horizontal directiondefect detecting unit 421, the vertical direction defect detecting unit422, the upper right direction defect detecting unit 423, and the lowerright direction defect detecting unit 424, and then one of the outputsof the detecting units 421 to 424 is selected and set as the correctiontarget using the texture direction information in the defective pixelposition selecting unit 431.

Alternatively, for example, in accordance with the texture directioninformation, the defect detecting unit performing the defect detectionin the same direction as the texture direction may be selectivelyoperated from the defect detecting units 421 to 424.

For example, as shown in FIG. 12, whether to operate any of the defectdetecting units 421 to 424 is determined using a switch 432 that iscontrolled in accordance with the texture direction information in thedefective pixel position selecting unit 431 so as to operate any one ofthe detecting units 421 to 424.

In the embodiment, although the example of using the first-orderdifferential value (gradient) in the defect detection is described, forexample, a second-order differential value (Laplacian) may be used inaddition to the first-order differential value (gradient).

7. Process of Defective Pixel Correcting Unit

The process of the defective pixel correcting unit 204 shown in FIG. 4will be described.

The defective pixel correcting unit 204 performs correction on thedefective pixel detected by the defective pixel detecting unit 203described above.

The correcting unit 204 determines the correction pixel value using theneighboring pixels on the basis of the texture direction determined bythe texture direction determining unit 202, with respect to the detecteddefective pixel.

As a method of determining the correction pixel value, various methodscan be applied, for example, a correction process referring to the pixelvalues of the neighboring pixels can be applied. For example, acorrection process such as a method of replacing the pixel value of thedefective pixel with the pixel value of the pixel closest to thedefective pixel position is performed in the texture direction. Inaddition, a process of selecting from 2×3 average values calculated bythe sharing pixel statistic calculating unit 411 may be applied in thetexture direction.

The present disclosure has been described in detail with reference tothe specific embodiment. However, it is obvious that a person skilled inthe art can correct and modify the embodiment within the scope whichdoes not deviate from the main concept of the present disclosure. Thatis, the embodiment of the present disclosure is disclosed as an example,and thus should not be interpreted as limiting. To determine the mainconcept of the present disclosure, it is preferable to refer to Claims.

The series of processes described in the specification may be performedby hardware, software, or combination of both. When the process isperformed by software, a program recording the process sequence may beinstalled and executed in a memory of a computer installed in dedicatedhardware, or the program may be installed and executed in ageneral-purpose computer which can perform various processes. Forexample, the program may be recorded in advance in a recording medium.In addition to the installation from the recording medium to thecomputer, the program may be received through a network such as LAN(Local Area Network) and the Internet and may be installed in arecording medium such as a built-in hard disk.

Various processes described in the specification may not only beperformed in time series according to the description, but also beperformed in parallel or individually in accordance with the performanceof the device performing the processes or as necessary. The system inthe specification is a logical group configuration of a plurality ofdevices, and the constituent devices are not limited to being providedin the same casing.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2011-006464 filed in theJapan Patent Office on Jan. 14, 2011, the entire contents of which arehereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image processing device comprising: a texture directiondetermining unit that determines a texture direction of an image; adefective pixel detecting unit that calculates a pixel value average foreach of pixel groups including a plurality of pixels, and detects adefective pixel position on the basis of difference information of thepixel value average according to an arrangement direction of the pixelgroups; and a correction unit that corrects, as a correction target, apixel value at the defective pixel position detected on the basis of thedifference information in the same pixel group arrangement direction asthe texture direction determined by the texture direction determiningunit.
 2. The image processing device according to claim 1, wherein thedefective pixel detecting unit calculates the pixel value average foreach pixel group including the plurality of pixels sharing a pixel valuereading circuit, and detects the defective pixel position on the basisof the difference information of the pixel value average according tothe arrangement direction of the pixel group.
 3. The image processingdevice according to claim 1, wherein the defective pixel detecting unitdetermines whether or not a pixel is a defective pixel in accordancewith a difference value from a reference value (safe_mW), the referencevalue being an average value of a plurality of pixel groups in a flatarea where a difference from the average value of the plurality of pixelgroups is small in a plurality of adjacent arrangements of pixel groupsin the same arrangement direction.
 4. The image processing deviceaccording to claim 1, wherein the texture direction determining unitperforms a process of determining one of four directions of horizontal,vertical, upper right, and lower right as the texture direction, andwherein the defective pixel detecting unit detects the defective pixelposition on the basis of the difference information of the pixel valueaverage according to the arrangement direction of the pixel groups inthe four directions of horizontal, vertical, upper right, and lowerright.
 5. The image processing device according to claim 1, wherein thetexture determining unit calculates a plurality of differential valuesbased on pixel values of pixels arranged in a predetermined directionincluded in a neighboring area centered on a pixel for attention, sortsthe plurality of differential values, selects only data with smallvalues, calculates statistics, and determines the texture direction onthe basis of comparison of the statistics.
 6. The image processingdevice according to claim 1, wherein the correction unit performs aprocess of determining the pixel value at the defective pixel positionwith reference to a neighboring pixel in the texture direction as areference pixel on the basis of the pixel value of the reference pixel.7. The image processing device according to claim 1, wherein thedefective pixel detecting unit calculates, as a process target, a pixelvalue average for each pixel group including a plurality of pixels inthe same pixel group arrangement direction as the texture directiondetermined by the texture direction determining unit, and detects thedefective pixel position on the basis of the difference information ofthe pixel value averages according to the arrangement direction of thepixel groups.
 8. An image processing method performed in an imageprocessing device, the method comprising: causing a texture directiondetermining unit to determine a texture direction of an image; causing adefective pixel detecting unit to calculate a pixel value average foreach of pixel groups including a plurality of pixels and to detect adefective pixel position on the basis of difference information of thepixel value average according to an arrangement direction of the pixelgroups; and causing a correction unit to correct, as a correctiontarget, a pixel value at the defective pixel position detected on thebasis of the difference information in the same pixel group arrangementdirection as the texture direction determined by the texture directiondetermining unit.
 9. A program for causing an image processing device toexecute an image process comprising: causing a texture directiondetermining unit to determine a texture direction of an image; causing adefective pixel detecting unit to calculate a pixel value average foreach of pixel groups including a plurality of pixels and to detect adefective pixel position on the basis of difference information of thepixel value average according to an arrangement direction of the pixelgroups; and causing a correction unit to correct, as a correctiontarget, the detective pixel position detected on the basis of thedifference information in the same pixel group arrangement direction asthe texture direction determined by the texture direction determiningunit.